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BACKGROUND  AND  PURPOSE 


It  is  Becoming  increasingly  evident  that  advances  in  technology 
(especially  microelectronics)  are  having  significant  effects  on  the 
place  of  men  in  Navy  weapon  systems--present  and  future.1  Decisions 
pertaining  to  the  allocation  of  system  functions  to  man  or  machine 
are  becoming  increasingly  difficult.  Such  decisions  could  be 
facilitated  by  the  use  of  a  cost-effectiveness  ratio,  this  ratio 
to  be  used  as  a  basic  formula  for  the  comparison  of  the  cost- 
effectiveness  of  functions  as  allocated  to  human  operation  versus 
automated  operation. 

This  bibliography  was  prepared  as  the  first  part  of  an  effort 
in  which  the  feasibility  of  developing  a  general  cost-effectiveness 
formula  is  being  examined.  The  major  areas  of  interest  represented 
in  this  bibliography  are: 

(A)  SYSTEM  COSTS,  including  a  representative  sample  of  factors 
which  would  relate  to  them. 

(B)  SYSTEM  EFFECTIVENESS,  including  reports  which  concern  the  cost- 
effectiveness  ratio. 

(C)  MAN/MACHINE  FUNCTION  ALLOCATION. 

BENEATH  THE  NUMBER  OF  EACH  BIBLIOGRAPHY  ENTRY  IS  A  LETTER  (A,  B, 

OR  C)  WHICH  INDICATES  THE  MAJOR  AREA(S)  OF  SUBJECT  MATTER,  CORRES¬ 
PONDING  TO  THE  ABOVE  THREE  AREAS  OF  INTEREST. 

An  addendum  includes  important  articles  which  became  available 
during  the  time  that  the  bibliography  was  in  press. 


^See  The  Impact  of  New  Technology  on  the  Navy 's  Human  Factor. 
Presentation  to  the  Chief  of  Naval  Material  Technical  Briefing, 

2  December  1964  and  to  the  Bureau  of  Naval  Personnel,  4  December 
1964,  Kaplan  I.  E.,  U.  S.  Naval  Personnel  Research  Activity, 

San  Diego,  California. 
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1.  Alchian,  Armen  A., 

(A)  Evidence,  The  Rand 

pp.  11,  (Unclassified) 

This  report  presents  some  data  on  the  reliability  of 
cost  estimates  obtained  from  several  sources.  It  was 
believed  that  studies  of  cost  estimates  and  their 
accuracy,  no  matter  what  the  type  of  item  being  pro¬ 
duced  or  operated,  would  shed  light  on  the  cost  estima¬ 
tion  reliability  problem.  The  range  of  unreliability 
detected  in  this  s4  dy  is  such  that  a  confidence  in 
being  able  to  bracket  the  true  costs  requires  bracketing 
values  in  the  ratio  of  about  2-1  (ratio  of  upper  limit 
to  lower  limit) . 

2.  Annotated  Bibliography  of  Hew  Developments  Research  Branch 
Studies,  Personnel  Research  Division,  Bureau  of  Naval  Personnel, 
January  1963,  PP1  ^5,  (Unclassified) 

3-  Arzigian,  Simon,  Methods  and  Problems  of  Computation  of 
(A)  Enlisted  Personnel  Costs,  U.  S.  Naval  Personnel  Research 

Activity,  Task  Assignment  301.01-01W,  PRAW  Report  No.  64- 16, 
February  1964,  pp.  59,  (Unclassified) 

Provides,  in  some  detail,  "an  explanation  of  the  method¬ 
ology  employed  in  the  computation  of  enlisted  personnel 
costs.  Sources  of  information  have  also  been  indicated 
and  an  analysis  of  problem  areas  which  are  encountered 
in  the  course  of  research  has  been  made." 

"Personnel  costs  discussed  in  this  report  are  by  no 
means  the  only  costs  incurred  by  the  Navy  in  the  personnel 
costing  area.  For  example,  additional  study  is  required 
in  the  area  of  training  costs,  particularly  for  the 
accurate  valuation  of  amortization  of  expensive  training 
equipment  and  the  costs  of  on-the-job  training  required 
after  school  to  bring  personnel  in  various  ratings  to 
an  acceptable  technical  level." 


Subsystem,  Thesis,  United  States  Naval  Postgraduate  School, 
1963,  pp.  37,  (Unclassified) 


"The  Polaris  Submarine  Weapons  System  is  presently  employing 
two  crews  per  submarine,  working  in  shifts  in  order  to 
maximize  the  time  on  station  for  each  submarine.  The  cost 
of  this  personnel  policy  as  compered  to  the  cost  of  using 
other  alternatives  is  of  current  interest.  The  purpose 
of  this  thesis  is  to  present  a  technique  for  making  this 
comparison.  The  actual  data  is  not  presented  here,  but 
hypothetical  examples  are  given." 


Reliability  of  Cost  Estimates  -  Some 
Corporation,  RM-4Bl,  30  October  1950, 
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5.  Barbour,  A.  A,,*  Firstman,  S.  I.;  &  Kamins,  M., 

(B)  Standardization  of  Automatic  Test  and  Checkout  Equipment; 
v'C)  A  Preliminary  Discussion,  The  Rand  Corporation,  RM-2^5, 

25  November  I960,  pp.  53>  (Unclassified) 

"This  report  is  intended  as  a  guide  in  the  evaluation  of 
the  technical  and  economic  feasibility  of  standardizing 
automatic  test  and  checkout  equipment.  Amounting  to  a 
"think  piece",  it  covers  what  the  authors  believe  to  be 
the  topics  pertinent -to  several  levels  of  standardization. 
It  is  not  a  partisan  study,  since  care  has  been  taken  to 
point  cut  the  problems  and  pitfalls  inherent  in  standard¬ 
ization,  and  it  includes  a  section  devoted  to  arguements 
against  standardization.  " 

6*  Barfoot,  Charles  B.,  A  Preliminary  Cost-Effectiveness  Handbook, 
(B)  Technical  Operations  Research,  Arlington,  Massachusetts, 
November  1963,  (Unclassified) 

This  preliminary  handbook  was  developed  for  use  by  United 
States  Army  Combat  Developments  Command  Field  Agencies 
and  AD  HOC  Study  Groups  in  the  preparation  of  cost- 
effectiveness  studies.  The  handbook  discusses  the  purpose 
of  cost-effectiveness  analysis,  general  methods  for  com¬ 
paring  alternative  systems,  and  the  major  elements  of  the 
analysis.  Two  examples  of  cost-effectiveness  evaluations 
are  presented.  General  references  and  a  bibliography  of 
published  sources  for  cost  data  and  of  publications  on 
systems  and  force  structure  costing  are  included. 

7.  Bell,  Chauncey  F.,  Cost-Effectiveness  Analysis  As  A  Management 
(B)  Tool,  Rand  Corporation,  Santa  Monica,  California,  October  1964, 
(Unclassified) 

The  potential  value  of  cost-effectiveness  analysis  as  a 
management  tool  is  discussed.  Such  analyses  can  be  useful 
because  they  suggest  an  examination  of  alternative  ways  of 
attaining  a  given  objective  or  goal,  because  they  focus 
attention  on  selecting  appropriate  objectives  and  relevant 
variables,  and  because  they  require  identification  and 
quantification  of  these  factors.  While  such  apparent 
precision  may  sometimes  lead  to  unwarranted  confidence  in 
the  results,  any  management  decision  is  affected  as  much 
or  more  by  such  short -comings.  Independent  of  cost- 
effectiveness  analysis,  on  balance,  then,  the  identifica¬ 
tion,  quantification,  and  systemization  of  cost- 
effectiveness  analyses  adds  to  the  likelihood  that  the 
resulting  judgement  decision  is  a  good  one. 
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8.  Bell,  Chauncey  F..;  Kamins;  &  Milton,  Determining  Economic 

(B)  Quantities  of  Maintenance  Resources:  A  Minuteman  Applica¬ 
tion,  The  Rand  Corporation,  Santa  Monica,  California, 

January  1963/  (Unclassified) 

This  memorandum  introduces  a  technique  for  computing 
manpower  and  equipment  requirements  that  takes  into 
account  three  critical  and  frequently  overlooked  factors . 
These  are:  the  randomness  of  the  failure  patterns — the 
uncertainty  of  the  time  any  particular  malfunction  will 
occur;  the  workshift  policy — when  maintenance  personnel 
are  on  duty;  and  the  cost-effectiveness  trade  off- -the 
marginal  increase  in  system  capability  weighed  against 
the  cost  of  providing  a  marginal  increase  in  resources. 

The  quantities  of  maintenance  personnel  and  ground  equip¬ 
ment  estimated  with  the  technique  presented  here  will 
economically  meet  anticipated  requirements  within  the 
accuracy  limitations  of  the  three  inputs  necessary  to 
the  computation.  These  are:  failure  rate  or  reliability, 
repair  time  or  maintainability,  and  cost. 

9.  Boren,  H.  E. ,  Jr.,  Individual  Weapon  System  Computer  Cost 

(A)  Model,  The  Rand  Corporation,  Santa  Monica,  California, 

RM  4l65PR,  AF49  638  700,  July  1964,  (Unclassified) 

This  memorandum  describes  the  operation  of  one  computer¬ 
ized  weapon  system  cost  model  used  by  the  Rand  Cost 
Analysis  Department  to  determine  resource  requirements 
for  individual  weapon  systems  that  consist  of  either 
aircraft,  aircraft  and  air -to- surface  missiles  (ASM'S), 
or  strategic  missiles  (for  personnel  requirements  only). 
Included  is  a  general  description  of  the  model  with 
particular  emphasis  on  the  personnel  subroutine,  which 
can  handle  eight  specific  types  of  weapon  systems  of 
the  Strategic  Air  Command,  the  Air  Defense  Command,  the 
Tactical  Air  Command,  and  the  Military  Air  Transport 
Service,  and  extrapolate  from  these  eight  weapon  systems 
72  varying  weapon  system  mixes  and  basing  situations. 

The  discussion  is  oriented  toward  describing  the  actual 
operation  of  the  model  rather  than  the  underlying  reasons 
for  the  methodology  used. 

10.  Bovaird,  R.  L.;  Goldman,  A.  S.»  &  Slattery,  T.  B. , 

(B)  Concepts  in  Operational  Support  Research,  TEMPO,  General 
Electric  Company,  hM  60TMP-70,  21  November  i960,  pp.  39, 
(Unclassified.) 
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This  paper  deals  with  concepts  involved  in  the  field  of 
"Operational  Support  of  Complex  Military  Systems". 
Operational  support,  as  used  here,  refers  to  the  process 
which  has  the  objective  of  keeping  system  availability 
(or  unavailability)  at  acceptable  levels.  An  attempt  is 
made  to  gather  into  a  cohesive  form  those  principles 
characterizing  the  new  science  which  is  envolving  from 
integrated  studies  on  reliability,  maintainability,  and 
logistics  problems.  The  concepts  presented  here  have  been 
developed  out  of  support  studies  of  several  current 
military  electronic  systems. 

11.  Brooks,  Samuel  H.,  Reliability  Cost /Effectiveness  A  Pilot 
(B)  Study,  Aerospace  Corporation,  San  Bernardino,  California, 

Final  Report. TDR  269S4853  1,  AK)4  695  269,  AK)4  695  469,  BSD 
TDR  64  91,  (Unclassified) 

A  mathematical  model  is  developed  and  explored  to  examine 
the  overall  relationship  between  effectiveness — in  terms 
of  reliability,  availability,  and  maintainability — and 
the  costs  to  be  allocated  and  apportioned  among  the 
reliability  efforts.  This  model  leads  to  an  estimate  of 
how  much  should  be  spent  on  reliability  during  design  and 
development  of  a  missile  system,  and  how  much  should  be 
spent  in  ensuring  the  reliability  and  availability  for 
each  individual  missile  during  its  operational  life.  A 
computer  program  was  developed  and  parameter  studies 
along  these  lines  were  initiated.  Requisite  optimization 
and  information  methods  are  discussed.  Preliminary 
indications  are  that  cost/effectiveness  modeling  is 
feasible  and  leads  to  methods  of  program  control.  Further 
studies  to  expand  these  results  are  recommended. 

12.  Clary,  'James  N.;  &  Arzigian;  Simon;  Personnel  Costs  for  Ten 

(A)  Selected  Ratings  (AC,  AE,  AQ,  AT,  AX,  ET,  FT  MM,  RD,  ST) 

Naval  Personnel  Research  Laboratory,  Washington,  D.  C.,  WRM 
No.  65-2,  April  1965,  (Unclassified) 

"Designed  to  meet  the  urgent  need  of  the  Secretary  of  the 
Navy’s  Task  Force  on  Military  Personnel  Retention  for 
personnel  costs  for  ten  selected  ratings.  Cost  information 
in  the  report  should  prove  useful  to  other  consumers  as 
well." 

"The  personnel  costs  presented  herein  are  based  on  the 
latest  available  information  and  reflect  all  known 
adjustments. " 
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13  .  Cole,  Roy  D.j  Keith/" Melvin  H.j  &  Seeley,  Richard  B., 

(B)  A  Light-Attack  Tactical  Weapon  System  Study  (U), 

Naval  Ordnance  Test  Station,  China  Lake,  California, 

January  i960,  (CONFIDENTIAL) 

14.  Connors,  Alonzo  J.;  Dye,  H.  '  j  Erler,  R.  L. ;  Joe,  R.  H. ; 

(A)  &  Martin, ,C..  P*. ,  Techniques  for  Predicting  R  and  D  Costs 
and  Schedules  of  Automatic  Flight  Control  Systems:  Volume 
1,  Aerodynamic  Vehicles.  Planning  Research  Corporation, 

Los  Angeles,  California,  Report  for  September  1962  - 
August  1963,  (Unclassified) 

The  objective  of  this  report  was  to  investigate  and 
formulate  techniques  for  predicting  the  development 
cost  and  development  time  of  automatic  flight  control 
systems  in  aerodynamic ally  supported  vehicles — both 
manned  and  unmanned.  An  empirical  approach  is  taken, 
using  relevant  historical  data  on  25  automatic  flight 
control  systems.  The  predictive  methodology  is  formu¬ 
lated  using  two  types  of  statistical  estimation  models. 

The  resultant  predictive  techniques  are  expressed  as 
simple  algebraic  equations  that  can  be  evaluated  to 
obtain  cost  and  time  estimates  for  a  proposed  flight 
control  system. 

is.  Cooper,  Joel,  "An  Integrated  Approach  to  System  Maintainability", 

(B)  in  F.  L.  Andenbrandt  (ed.),  Electronic  Maintainability,  Volume 

(c)  3,  Reinhold,  New  York,  i960,  (Unclassified) 

"Since  a  high  percentage  of  malfunctions  are  initiated  by 
humans,  a  method  for  estimating  the  'human  reliability' 
of  a  system  would  be  useful  during  the  planning  and 
development  sequence.  It  is  proposed  that  classifying 
maintenance  operations  and  collecting  data  on  time, 
accuracy,  and  variability  in  performing  them  would  be 
useful.  Such  data  could  be  applied  for  example,  to  the 
man-machine  allocation  problem:  If  human  response 
variability  for  a  set  of  operations  is  too  great,  then 
that  group  of  tasks  becomes  a  good  candidate  for  mechani¬ 
zation  cr  procedural  revision. 

16.  Deprotine,  P.  R.,  Characteristics  of  ABM  Systems  Optimized 

(A)  for  Defense  Aged  ns t  Small  Attacks.  Douglas  Aircraft  Company, 
Incorporated,  Santa  Monica,  California,  SM48055,  SD247, 

November  1964,  (Unclassified) 
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This  appendix  presents  the  cost  breakdown  of  all  System-A 
deployments  considered  in  detail.  The  costs  are  presented 
in  two  ways.  First,  the  cost  of  individual  batteries  or 
radar  sites  is  presented,  followed  by  a  summation  of  the 
total  investment  for  all  sites  in  a  specific  system. 
Second,  the  annual  operating  and  maintenance  costs  are 
listed  per  battery  and  radar  site,  followed  by  a  summation 
for  total  system  annual  and  5-year  operating  and  mainte¬ 
nance  costs.  Each  section  presents  the  data  in  both  the 
format  provided  in  the  ARPA  cost  guidelines  and  in  the 
cost  model  format .  Taking  the  RDT&E  category  as  an 
example,  the  first  sheet,  starting  with  preliminary  re¬ 
search  and  design  studies  lists  all  elements  of  the  basic 
cost  model.  The  following  sheet,  headed  RDT&E  format, 
rearranges  these  elements  into  the  ARPA  cost  guideline 
format.  This  dual  arrangement  continues  for  investment 
and  operating  cost  categories.  The  cost  model  presenta¬ 
tion  is  retained  because  its  greater  level  of  detail 
permits  the  reader  to  make  his  own  analysis  of  any  major 
cost  area  of  interest .  The  index  is  arranged  to  permit 
easy  retrieval  of  any  category  or  element  of  cost. 


Deprotine,  P.  R.,  Characteristics  of  ABM  Systems  Optimized 
for  Defense  Against  Small  Attacks,  Volume  III,  Technical 

endixes.  Part  6,  Appendix  G3,  System  B  Cost  Substantiation 
,  Douglas  Aircraft  Company,  Incorporated,  Santa  Monica, 
California,  SM48055>  SD247,  November  1964,  (Unclassified) 


This  Appendix  presents  the  cost  breakdown  of  all  System-B 
deployments  considered  in  detail.  The  costs  are  presented 
in  two  ways.  First,  the  cost  of  individual  batteries  or 
radar  sites  is  presented,  followed  by  a  summation  of  the 
total  investment  for  all  sites  in  a  specific  system. 
Second,  the  annual  operating  and  maintenance  costs  are 
listed  per  battery  and  radar  site,  followed  by  a  summation 
for  total  system  annual  for  5-year  operating  and  mainte¬ 
nance  costs. 


18.  Deprotine,  P.  R.,  Characteristics  of  ABM  Systems  Optimized 
(A)  for  Defense  Against  Small  Attacks,  Volume  III,  Technical 


ndixes.  Part  7,  Appendix 


E£ 


System  C  Cost  Substantiation, 


Douglas  Aircraft  Company,  Incorporated,  Santa  Monica, 
California,  November  1964,  (Unclassified) 


This  appendix  presents  the  cost  breakdown  of  all  System-C 
deployments  considered  in  detail.  The  costs  are  presented 
in  two  ways,  first  the  coBt  of  individual  batteries  or 
radar  sites  is  presented,  followed  by  a  summation  of  the 
total  investment  for  all  sites  in  a  specific  system. 
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Second^  the  annual  operating  and  maintenance  costs  are 
listed  per  battery  and  radar  site,  followed  by  a  summation 
for  total  system  annual  for  5-year  operating  and  mainte¬ 
nance  costs. 

19  .  Enlisted  Personnel  Costs  For  Use  In  SEA  HAWK  Cost  Effectiveness 

(A)  Comparisons  (Eirst  Report).  New  Developments  Research  Branch, 

Personnel  Research  Division,  Bureau  of  Naval  Personnel, 
Washington,  D.  C.,  Report  No.  ND  64-63,  May  1964,  (Unclassified) 

Provides  ".  .  information  on  personnel  costs  to  be  used 

in  cost  effectiveness  comparisons  being  conducted  for  the 
SEA  HAWK  project."  It  .  .  .  "utilizes  existing  BUPERS 
cost  methodology  that  has  been  developed  in  earlier  reports, 
and  in  some  instances  additional  consideration". 

"This  SEA  HAWK  personnel  cost  report  is  divided  into  two 
major  parts.  The  first  part  presents  in  detail  the  method 
of  computation  of  personnel  cost  figures  and  discusses 
ways  of  implementing  personnel  costs  in  SEA  HAWK  Program 
evaluations.  The  second  part  presents  cost  data  sheets 
for  twenty-seven  ratings." 

20  .  Enthoven,  Alain  C . ,  The  Economics  of  Navy  Pay,  The  Rand 

(A)  Corporation,  P-1051,  2  Apri'  1957,  PP-  lo,  (Unclassified) 

The  occasion  for  this  discussion  was  the  appearance  of 
the  Cordiner  report  on  the  military  pay  structure. 

The  author  states  that  the  recommendations  made  in  the 
Cordiner  report  are  a  step  in  the  right  direction,  and 
they  represent  an  attempt  to  readjust  salary  scales  in 
accordance  with  supply  and  demand  and  productivity 
considerations.  However,  as  an  economist,  the  author 
would  like  to  see  more  reliance  placed  upon  a  flexible 
system  of  reenlistment  bonuses.  He  states  that  such  a 
system  would  be  particularly  desirable  for  the  Navy. 

21.  Enthoven,  Alain  C.,  The  Mathematics  of  Military  Pay,  Rev. 

(A)  ed.,  The  Rand  Corporation,  P-1100,  11  November  1957,  PP*  30, 

(B)  (UncxasBified) 

The  purpose  of  this  paper  is  to  explain  the  mathematical 
theory  of  maximization  of  military  effectiveness  within 
the  limitations  of  a  fixed  budget  and  to  show  that  the 
theory  has  an  appropriate  application  in  the  pay  question. 
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22.  Firstman,  Sidney  I.,  Hov  Much  Automaticity  for  Checkout 
(C)  Equipment ,  The  Rand  Corporation,  P-18&7,  4  January  I960, 
pp.  24,  (Unclassified) 

The  author  states  that  the  degree  of  automaticity  with 
which  an  automatic  checkout  system  operates  is  determined 
primarily  by  the  method  of  programming  (or  control) 
employed,  i.e.,  whether,  for  example,  a  digital  computer 
or  a  paper  tape  control  system  with  manual  decision  is 
employed.  And  since  the  nature  of  the  programmer  deter¬ 
mines,  to  a  large  extent,  the  nature  of  the  checkout 
system  (and  vice  versus),  this  paper  discusses  some  of 
the  relevant  differences  between  the  two  generic  types 
of  automatic  programmers — internally  and  externally 
programmed — that  should  be  considered  when  determining 
the  degree  of  automaticity  to  build  into  an  automatic 
checkout  system  for  any  particular  application. 


23«  Firstman,  Sidney  I.,  Some  Limitations  of  Automatic  Test 
(C)  Equipment,  The  Rand  Corporation,  P-2319,  22  May  1961,  pp.  6, 

(B)  (Unclassified) 

This  article  discusses  some  of  the  limitations  of 
automatic  test  equipment  (ATE).  The  limitations  are 
broken  into  the  following  four  categories:  (l)  limitations 
surrounding  automatic  testing  per  se;  (2)  those  most 
clearly  associated  with  ATE-prime  equipment  interactions; 
(3)  those  associated  with  ATE  operating  in  its  environment; 
and  (k)  those  concerned  primarily  witv  man-machine 
activities. 


Firstman,  S.  I.;  Barbour,  A.  A.;  Brom,  J.  R.;  Jordan,  N.; 
Kamins,  M.;  Meyer,  K.  H.;  &  Voosen,  B.  J.,  An  Omnibus  of 

Briefing  Papers  on  Analysis  of  Automatic  Checkout  Equipment 
and  Aids  to  Its  Design,  The  Rand  Corporation,  RM-2750, 

12  June  1961,  pp.  158,  (Unclassified) 

"This  memorandum  is  a  summary  briefing,  together  with 
shorter  back-up  briefings,  on  a  RAND  study  of  automatic 
checkout  equipment  and  aids  to  its  design  (Project  ACE) . 
The  material,  essentially  unchanged,  has  been  gathered 
in  this  format  to  make  ACE  results  available  quickly  to 
persons  vho  are  concerned  with  automatic  checkout  equip¬ 
ment  but  vho  have  not  heard  the  presentations. 

The  memorandum  opens  with  a  summary  of  RAND’s  Project 
ACE,  including  a  checklist  of  operations  features. 

This  is  followed  by  nine  briefing  papers  on  subordinate 
aspects.  Detailed  back-up  material  is  cited  where 
appropriate.  A  bibliography  of  ACE  publications  released 
or  in  process  is  provided  on  p.  vii." 
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25*  Firstman,  Sidney  I.;  &  Jordan,  Nehemiah,  Operational  and 

(B)  Human  Factors  in  Planning  Automated  Man-Machine  Checkout 

(C)  Systems ,  The  Rand  Corporation,  Memorandum  RM-2835>  April  1962, 
pp.  60,  (Unclassified) 

"This  Memorandum,  aimed  at  improvement  of  planning  and 
designing  of  automated  checkout  and  test  equipment  (ACE) 
for  military  systems,  is  part  of  a  larger  RAND  study  of 
automatic  checkout  equipment. 

It  is  based  in  large  measure  on  an  extensive  survey  of 
automated  checkout  and  test  equipment  as  it  has  been  used 
in  the  field,  supplemented  by  study  of  human  factors 
relevant  to  both  the  process  of  automating  such  equipment, 
and  the  subsequent  operation  of  the  ACE,  in  furtherance 
of  overall  effectiveness  of  military  systems." 

26.  Fisher,  G.  H.,  Weapon  System  Cost  Analysis,  The  Rand  Corporation, 

(A)  Santa  Monica,  California,  Report  No.  Po23>  (Unclassified) 

This  paper  is  concerned  with  how  cost  is  to  be  estimated 
and  taken  into  account  in  an  analysis  of  weapon  systems. 

It  is  assumed  that  the  analyst  is  dealing  with  that  class 
of  systems  analysis  problems  where  cost  considerations 
are  relevant. 

27.  Fitts,  P.  M.  (ed.),  Human  Engineering  for  an  Effective  Air- 
(C)  Navigation  Air  Traffic-Control  System,  Ohio’  State  University 

Research  Foundation,  December  1950,  (Unclassified) 

The  following  four  alternative  man-machine  systems  are 
described:  (l)  fully  automatic  control,  (2)  automatic 
control  with  human  monitoring,  (3)  semiautomatic  control 
supplemented  by  human  performance  of  critical  functions, 
and  (b)  primary  control  by  human  operators  who  would  be 
assisted  by  effective  data-analysis,  data-transmission, 
and  data-di splay  equipment. 

The  abilities  in  which  humanB  (machines)  surpass  machines 
(humans)  are  also  given. 

28.  Fox,  Peter  D.,  A  Theory  of  Cost-Effectiveness  for  Military 

(B)  Systems  Analysis,  Operations  Research,  Volume  13>  No*  2, 
March-April  1965,  pp .  191-200,  ( Unc lassif led) 

This  paper  presents  a  theoretical  basis  for  cost- 
effectiveness  analysis.  It  is  argued  that,  frequently, 
a  range  of  effectiveness  or  cost  levels  may  be  acceptable 
to  whoever  must  ultimately  decide  which  military  system 
(if  any  should  be  acquired.  The  function  of  the  analyst 


is  to  present  a  schedule  of  alternatives  and  not  to 
optimize  in  the  sense  that  he  recommends  the  selection 
of  a  particular  alternative.  The  formulation  of  the 
schedule  is  discussed  where  the  cost  and  effectiveness 
associated  with  each  alternative  are  viewed  as  random 
variables.  The  paper  concludes  with  some  general 
observations  relating  to  military  system  selection. 

29.  Franks,  P.  E.j  &  Furnish.  C.  W.,  Automated  Maintenance: 

(B)  Theory,  Practice,  and  Implications  for  Training,  WADD 

(C)  Technical  Report  60-412,  August  I960,  pp.  23,  (Unclassified) 

"The  automatic  checkout  concept  may  be  Justified  insofar 
as  it  (l)  reduces  skilled  manpower  needs,  or  (2)  serves 
to  achieve  an  extremely  short  operational  reaction  time. 

A  successful  checkout  system  must  be  multipurpose, 
accurate,  efficient  (in  terms  of  testing  time),  seldom 
fail,  and  have  a  quick  change  capability  while  keeping 
costs  within  reasonable  limits. 

Assuming  that  ACE  will  soon  be  in  use  and  time  is  short 
to  prepare  for  its  impact,  the  anticipated  effects  of 
ACE  on  personnel  skill  levels,  motivation,  logistics, 
cost,  system  accuracy,  and  training  are  discussed." 

30.  Freed,  Alvyn  M.,  Human  Interactions  in  Man-Machine  Systems, 

OO  Human  Factors,  Vol.  4,  No .  6, December  1962,  PP.  389-396, 

(c)  (Unclassified) 

"Human  interactions  play  a  vital  role  in  the  reliability 
of  man-machine  systems.  Techniques  are  necessary  to 
insure  that  those  which  Occur  do  so  because  they  are  so 
designed  and  planned.  A  tentative  approach  to  providing 
lists  of  such  interactions,  and  ways  of  defining, 
labelling  and  measuring  them  are  suggested  as  basic  to 
design  input.  Methods  for  isolating  units  of  interactive 
behavior  are  proposed  and  samples  of  Bystem  behavior  and 
their  respective  activities  described  in  terms  of  actions 
and  reactions.  The  need  for  definition  and  labelling 
of  activities  couched  in  operational  terms  is  emphasized 
in  the  interest  of  design,  training  and  measurement  of 
human  interactions  in  man-machine  systems.  Techniques 
for  accomplishing  these  steps  are  suggested." 

31..  Gagliardi,  U.  0.;  Kaplan,  J.;  &  Vallerie,  L.  L.,  Man- 

Computer  Systems  and  Allocation  of  Resources  Problems, 

Dunlap  and  Associates,  Incorporated,  Contract  No.  Nonr-3602(00) , 
January  1964,  (Unclassified) 


The  work  reported  concerns  the  observation  of  problem¬ 
solving  behavior  exhibited  by  subjects  who  were  given 
an  allocation-of -resources  task.  The  tasx  was  to  deploy 
Polaris-like  weapon  systems  against  a  given  target  system 
under  stated  constrainst.  While  the  task  is  formulable 
as  an  integer  linear  programming  problem,  the  subjects 
solved  it  by  resorting  to  heuristic  procedures.  These 
procedures,  as  well  as  the  solutions  produced,  seem  to 
indicate  that  a  problem  solver  may  encounter  consider¬ 
able  difficulty  in  uncovering  the  ordering  of  decision 
alternatives,  if  this  ordering  is  a  partial  one. 

Following  the  study  of  unaided  performance,  two  distinct 
computer-aid  concepts  were  developed  and  tested. 

32.  Geisler,  M.  A. ,  Man- Machine  Simulation  Progress,  The  Rand 

(B)  Corporation,  f-2086,  23  August  i960,  pp. 10,  (Unclassified) 

This  paper  discusses  a  technique  which  has  been  largely 
developed  through  use  in  logistics  research.  T1  3  tech¬ 
nique  has  been  applied  to  studies  of  large  logistics 
management  systems  in  which  decision-making  under  un¬ 
certainty  is  required.  The  procedure  is  to  build  man- 
machine  simulations,  and  to  use  them  in  experimental 
situations.  The  output  is  a  description  of  decision 
rules,  information  flows,  and  an  organizational  structure 
that  improve  the  cost  and  effectiveness  of  the  logistics 
system.  Man  is  used  in  these  simulations  for  his  learn¬ 
ing,  adaptiveness,  and  flexibility. 

33.  Goldman,  A.  S.,  Introduction  to  the  Economic  Theory  of  System 

(A)  Development  in  Operational  Support ,  TEMPO,  General  Electric 

(B)  Company,  RM  62TMP-39,  1  May  1962,  pp.  59,  (Unclassified) 

"The  prime  purpose  of  this  report  is  to  present  decision 
making  tools,  derived  from  economic  theory,  to  assist 
in  the  development  of  complex  military  systems.  The 
emphasis  is  on  the  operational  support  aspects 
(reliability-maintainability-availability)  of  system 
design  but  the  reader  will  also  find  that  the  philosophy 
applies  to  other  areas  as  well  (e.g.,  performance  trade 
offs).  A  secondary,  but  no  less  important,  purpose  is 
to  present  a  primer  of  production  economics  for  reference 
by  engineers  in  various  phases  of  product  development." 
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34.  Gordon,  T.  J.;  &  Helmer,  0.,  Report  on  a  Long-Range 

(B)  Forecasting  Study,  The  Rand  Corporation,  Santa  Monica, 

(C)  California,  September  1964,  (Unclassified) 

This  report  describes  an  experimental  trend  predicting 
exercise  covering  a  period  extending  as  far  as  fifty 
years  into  the  future.  The  experiment  used  a  sequence 
of  questionnaires  to  elicit  predictions  from  individual 
experts  in  six  broad  areas:  scientific  breakthroughs, 
population  growth,  automation,  space  progress,  probability 
and  prevention  of  war,  and  future  weapon  systems.  Results 
of  the  experiment  illuminate  a  number  of  points:  The 
contents  of  the  predictions  themselves,  the  bases  on  which 
respondents  claimed  their  predictions  were  made,  the  spread 
of  expert  views,  the  convergence  of  views  following  data 
feedback,  the  expert's  critiques  of  each  other's  views, 
and  not  least  of  all,  the  weaknesses  of  the  method  and 
the  possible  means  for  improving  it. 

35-  Grodsky,  M.  A.j  Levy,  G.  W.j  &  Sorkin,  R.  D. ,  A  Model  of  Human 

(B)  Maintenance  Behavior-I:  Development  of  the  Problem  and  Evidence 

(C)  for  a  Model,  The  Martin  Company  Engineering  Report  11133 >  19^0 , 
(Unclassified) 

Four  criteria  are  suggested  for  studying  the  advantages  of 
automatic  or  manned  modes  for  space  flight:  payload 
weight,  economy  (overall  operational  costs),  utility  of 
man,  and  mission  effectiveness. 


Previous  studies  indicated  that  application  of  these 
criteria  justified  a  manned  vehicle.  "It  is  apparent  that 
there  is  a  definite  per  cent  weight  advantage  in  using  a 
manned  mode  of  maintenance  as  opposed  to  an  automatic  mode 
of  maintenance.  It  should  be  noted  that  the  increased 
weight  in  the  manned  mode  of  maintenance  also  includes  the 
weight  required  to  support  men  within  the  system."  These 
savings  may  be  attenuated  in  the  actual  case  because 
equivalent  or  greater  cost  expenditures  would  be  necessi¬ 
tated  by  the  selection  and  training  of  personnel  to  perform 
maintenance. 

36.  Haythorn,  W.  W.,  Human  Factors  in  Systems  Research,  The  Rand 

(C)  Corporation,  P-2337,  7  June  1961,  pp.  32,  (Unclassified) 

In  the  research  described  in  this  paper,  the  primary 
methodological  device  is  simulation  combining  symbolic 
representations  of  some  aspects  of  systems,  realized  on 
a  digital  computer,  with  individual  decision  makers  urawn 


from  the  population  from  which  the  ultimate  users  of  the 
system  are  drawn.  The  organization  of  the  paper  is 
based  on  the  sequence  of  personnel  subsystem  considera¬ 
tions  in  the  system  development  process  including: 

(l)  the  allocation  of  tasks  to  men  and  machines,  (2)  the 
estimation  of  manpower  requirements  by  skill  types  and 
levels,  (3)  systems  training,  (4)  decision  analysis, 

(5)  the  use  of  simulation  in  systems  design,  and  (6)  more 
basic  human  factors  research. 


37.  Haythorn,  W.  W. ,  Maintenance  Scheduling  Decisions  and  the 
(B)  Importance  of  Information,  The  Rand  Corporation,  P-2302, 

9  May  196l,  pp.  23,  (Unclassified) 

A  procedure  for  estimating  maintenance  personnel  require¬ 
ments  by  Air  Force  specialty  code,  using  information  re¬ 
garding  the  reliability  of  the  hardware  under  a  variety 
of  conditions  and  the  possible  operational  and  maintenance 
policies  to  be  adopted,  was  developed.  These  two  aspects 
of  the  study  and  their  impact  on  overall  system  cost  and 
effectiveness  were  discussed. 

The  application  of  the  information  system  and  the  utiliza¬ 
tion  of  the  maintenance  personnel  requirements  estimation 
technique  were  instrumental  in  increasing  missile  alert 
status  by  approximately  15  per  cent  and  reducing  total 
system  cost  by  like  amount. 

38.  Heinemann,  Robert  W.,  An  Objective  Approach  to  Program 
(b)  Planning,  Ammunition  Engineering  Directorate,  Picatinny 

Arsenal,  Dover,  New  Jersey,  TM13^5,  December  1964  (Unclassified) 

Research  and  development  funds  expended  on  systems  which 
never  reach  the  field  waste  huge  sums  of  money  every 
yeur.  This  fact  is  accepted  as  inevitable  on  the  basis 
that  only  a  limited  number  of  the  completed  studies  will 
meet  with  sufficient  success  to  permit  field  use.  The 
reasons  for  dropping  the  other  studies  involve  factors 
such  as  complexity  of  design,  manufacture  or  operation, 
cost,  marginal— or  no  improvement,  unreliability,  safety 
problems,  lack  of  requirements,  lack  of  funds,  unfeasi¬ 
bility  or  aborted  studies.  This  report  outlines  a  pro¬ 
gram  which,  if  properly  developed,  would  yield  the  required 
information  to  objectively  analyze  proposed  major  programs. 
Further  work  is  required  to  formulate  the  program  into  a 
workable  and  useful  management  tool. 
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(C) 


4o. 

(B) 


41. 

(A) 


42. 

(B) 

(C) 

^3. 

(B) 


Hoisman,  A.  J.;  &  Daitch,  A.  M. ,  Techniques  for  Relating 
Personnel  Performance  to  System  Effectiveness  Criteria:  A 
Critical  Reviev  of  the  Literature,  Dunlap  and  Associates, 
Incorporated  (Western  Operations),  Contract  No.  Nonr-43l4((X))> 
September  1964,  pp.  45,  (Unclassified) 

Howard  W.  J. ;  &  Goldman,  A.  S.,  Application  of  Some  Economic 
Concepts  to  System  Operational  Availability,  TEMPO,  General 
Electric  Company,  RM  62TMP-7,  1  June  1962,  pp.  4l, 
(Unclassified) 

Subject  to  performance  requirements,  systems  designers 
and  management  are  concerned  with  either  (l)  allocating 
a  fixed  budget  between  reliability  and  maintainability 
in  order  to  maximize  availability,  or  (2)  attaining  a 
required  availability  at  minimum  cost.  By  means  of  dis¬ 
cussion  and  example,  the  problem  of  choosing  among 
alternatives  is  investigated  relative  to  achieving 
desired  levels  of  system  availability. 

Howell,  Richard  P.,  Method  for  Estimating  Manufacturing 
Facility  Requirements,  Stanford  Research  Institute,  Menlo 
Park,  California,  (Unclassified) 

The  objective  of  this  research  was  to  derive  a  means  of 
making  or  evaluating  an  early  estimate  of  facility  re¬ 
quirements  for  contemplated  weapon  systems.  Given  the 
maximum  level  of  annual  expenditure  anticipated  for  the 
procurement  of  the  hardware  of  the  system.  The  method 
described  in  this  report  was  developed  on  the  basis  of 
data  obtained  from  authoritative  Published  sources. 
Because  of  the  limits  of  time  and  money  available, 
efforts  were  not  made  to  refine  the  individual  companies, 
however,  since  the  results  are  accompanied  by  an 
expression  of  the  probable  range  of  values  that  might 
be  expected,  the  user  has  at  his  disposal  a  means  for 
determining  whether  the  results  are  accurate  enough  for 
his  requirements. 

Human  Factors  in  Automatic  Checkout  Equipment:  An  Annotated 
Bibliography,  The  Rand  Corporation,  Memorandum  RM- 27 56 -PR. 
March  1962,  pp.  88,  (Unclassified) 

Investigation  of  the  Relationships  of  Operational  Reliability, 
Development  Cost,  and  Development  Time  of  Selected  Guided 
Missiles  (U),  Planning  Research  Corporation,  Lob  Angeles. 
California,  Final  Report,  March  1963,  (CONFIDENTIAL) 


44.  Jannsen,  T.  J.;  Glazer,  H.;  &  Des  Roches,  J.  C.,  User 's 

(A)  Manual  for  the  Computerized  Electronic  System  Cost  Model, 
MITRE  Corporation,  Bedford,  Massachusetts,  October,  1964, 
(Unclassified) 

The  economic  factors  department  of  MITRE  has  developed 
a  computerized  electronic  system  cost  model  as  part  of 
its  work  in  system  cost  methodology.  This  document 
described  the  Initial  version  of  the  model  and  gives 
instructions  for  the  electronic  data  processing  pro¬ 
cedures,  including  presentation  of  the  key  punch  input 
forms,  computer  output  formats,  and  a  description  of 
the  computer  program  logic.  The  model,  currently 
operational,  has  been  programmed  for  the  IBM  7090  com¬ 
puter  requires  a  32,768  register  core  memory.  The 
objectives  of  this  manual  are  to  provide  a  primer  for 
the  cost  analyst,  and  a  reference  manual  for  the 
detailed  operations  on  the  use  of  the  model. 

45  .  Jordan,  Nehemiah,  Allocation  of  Functions  Between  Man  and 

(C)  Machines  in  Automated  Systems,  Journal  of  Applied 

Psychology,  Volume  47,  No.  3>  June  1963,  pp.  l6l-l65, 
(Unclassified) 

"Fitts  (in  an  article  in  1951)  recommended  that  man  be 
compared  to  machines  and  be  chosen  for  those  functions 
which  he  does  better  than  machines  and  vice  versa.  To 
do  so  is  wrong.;  when  we  can  compare  a  man  to  a  machine, 
we  find  that  we  can  also  build  a  machine  for  the  function 
involved.  Hence  the  lack  of  progress.  Men  and  machines 
are  complementary,  rather  than  comparable.  Once  the 
problem  is  so  reformulated,  new  ways  of  thinking,  which 
appear  to  be  promising,  open  up." 

46  .  Jordan,  Nehemiah,  Human  Factors  Aspects  in  Maintainability, 

(B)  The  Rand  Corporation,  P-2459*  October  1961,  pp.  11, 

(C)  (Unclassified) 


This  article  takes  a  look  at  the  broad  area  of  maintain¬ 
ability  as  discussed  by  human  factors  scientists  and 
engineers.  This  area  is  further  subdivided  into  the 
three  major  interdependent  sub-areas  of:  design  for 
maintainability;  training  the  maintenance  man  to  be 
able  to  assume  the  responsibility  for  maintaining  the 
equipment;  and  the  tools,  equipment  and/or  aids  that 
are  necessary  to  enable  the  maintenance  man  to  meet 
his  responsibilities.  Each  of  these  areas  is  considered 
in  turn  as  they  are  related  to  human  factors  problems. 
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47.  Jordan,  Nehemiah,  The  Allocation  of  Functions  Between  Man  and 

(C)  Machines  in  Automated  Systems,  The  Rand  Corporation,  P-2310, 

lS  May  1961,  pp.  11,  (Unclassified) 

This  article  briefly  reviews  the  history  and  problems 
associated  with  the  study  of  allocation  of  functions. 

The  author  also  discusses  the  allocation  problem  within 
the  framework  of  automated  systems. 

48.  Kidd,  Jerry  S.,  A  New  Look  At  System  Research  And  Analysis, 

(B)  Human  Factors,  Vol.  4,  No. 4,  August  1962,  pp.  209-216, 

(Unclassified) 

This  report  is  an  attempt  to  integrate  some  conceptual 
and  methodological  divergencies  in  man-machine  system 
research.  A  conceptual  format  and  a  procedure  for 
input  analysis  are  proposed  which  are  derived  from  a 
cybernetics  model.  The  format  is  suggested  as  a  means 
to  organize  theoretical  propositions.  Some  problems 
related  to  real-time  simulation  as  a  research  method 
for  system  research  are  considered  and  an  approach  to 
methods  improvement  is  discussed. 

49.  Lambert,  C.  G. ,  Index  of  Published  Papers,  System  Development 
Corporation,  SP-O/OOO/OI,  1  June  1962,  pp.  60,  (Unclassified) 

50.  Mann,  L.  0.;  Primas,  W.  H.j  &  Jackson,  R.  J.,  Enlisted  Personnel 
Costs  for  Use  in  ASW  Surface  Ship  Systems  Cost-Effectiveness 
Comparisons,  Personnel  Research  Division,  Bureau  of  Naval 
Personnel,  Washington,  D.  C.,  Report  No.  ND  65-40,  December 
1964,  (Unclassified) 

"...  Presents  personnel  costs  for  the  twenty-seven 
ratings  envisioned  as  being  utilized  in  ASW  surface 
ship  systems  and  related  subsystems. 

The  personnel  cost  data  provided  are  presented  in  Section 
II  in  two  tabular  listings;  (l)  cumulative  cost  per  pay 
grade  for  each  rating  reported  and  (2)  cumulative  cost 
per  year  per  pay  grade  for  each  rating.  These  listings 
differ  considerably  from  the  first  and  second  enlistment 
type  costs  employed  in  the  previous  SEA  HAWK  personnel 
cost  reports.  '  The  latter  cost  listings  did  not  allow 
the  conciseness  and  flexibility  required  for  cost- 
effectiveness  comparisons.  The  method  employed  in  this 
memorandum  has  made  an  attempt  to  include  all  known 
cost  elements  paid  by  the  government  in  support  of 
enlisted  personnel." 
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I  .  51.  Marshall,  A.  W.;  &  Meckling,  W.  H. ,  Predictability  of  the 

j  ( B)  Costs,  Time  and  Success  of  Development,  Rev .  ed . ,  The  Rand 

l  Corporation,  P-1&21,  11  December  1959  >  PP-  24, 

j  (Unclassified) 

In  its  role  as  advisor  to  the  Air  Force  on  research  and 
I  development  policies  RAND  is  continually  forced  to  make 

\  use  of  predictions  about  particular  development  projects. 

,  Inevitably  there  has  been  concern  over  the  confidence 

j  which  can  be  attached  to  such  predictions.  This  paper 

>:  presents  the  results  of  some  recent  research  into  the 

I  extent  and  nature  of  the  uncertainty  in  new  developments. 


52.  McGrath,  Joseph  E.j  &  Nordie,  Peter  G. ,  Synthesis  and 

(B)  Comparison  of  System  Research  Methods,  Human  Sciences  Research, 
Incorporated,  Contract  No.  2525(00) ,  Report  No.  9,  February 
i960,  pp.  143,  (Unclassified) 

In  1958>  Human  Sciences  Research,  Incorporated  initiated 
a  research  program  aimed  at  synthesis  of  system  research 
methodology  under  sponsorship  of  Personnel  and  Training 
Branch,  Psychological  Sciences  Division,  Office  of  Naval 
Research. 

The  central  objective  of  Phase  I  was  to  review  methodologies 
of  system  research  studies  to  attempt  to  construct  a  use¬ 
ful  set  of  concepts  for  organizing  and  comparing  research 
methods,  and  to  evaluate  the  potential  fruitfulness  of 
such  an  approach  for  providing  a  synthesis  of  system 
research  methods.  Phase  II  was  basically  an  extension  of 
the  procedures  used  in  the  Phase  I  feasibility  study,  in 
order  to  modify  and  elaborate  the  concepts  which  had  been 
developed. 

53.  McGrath,  J.  E.j  &  Nordie,  P.  G.,  "Theory  of  Allocation  of 

(C)  Functions  to  Man  and  Machines",  Synthesis  and  Comparison  of 
System  Research  Methods,  Human  Sciences  Research  Incorporates, 
Report  HSR-RR-60/l-SM,  February,  i960,  (Unclassified) 

".  .  .a  two-part  approach  is  seen  as  being  required  for 
a  solution  to  the  allocation  problem.  First,  a  common, 
descriptive,  quantitative  language  must  be  developed 
which  can  be  applied  to  man  and  machine  capabilities. 

Second,  a  logic  of  allocation  decisions  must  be  formulated. 
The  two  are  clearly  interdependent.  The  problem  will 
receive  major  emphasis  in  Phase  III  of  this  research 
program." 
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McGrath,  Joseph  E.;  Nordlie,  Peter  G.;  &  Vaughan,  W.  S.,  Jr., 
A  Systematic  Framework  for  Comparison  of  System  Research 
Methods,  Human  Sciences  Research,  Incorporated,  Report  No.  1, 
November  1959>  PP*  62,  (Unclassified) 

The  central  aims  of  the  program  are  to  systematically 
review  and  integrate  the  methods  which  have  been  employed 
in  studies  of  complex  man-machine  systems;  to  compare 
alternative  methods  and  procedures  which  have  been  used 
to  carry  out  various  steps  of  such  system  studies;  and  to 
determine  the  major  methodological  weaknesses  requiring 
concentrated  research  effort.  Subsidiary  aims  are  to 
develop  a  frame  of  reference  and  a  common  language  within 
the  system  research  domain  to  aid  communication  among 
scientists  from  diverse  disciplines,  and  to  suggest  means 
of  applying  existing  methods  to  new  problem  areas. 

McKendry,  James  M.j  &  Harrison,  Paul  C.,  Assessing  Human 
Factors  Requirements  in  the  Test  and  Evaluation  Stage  of 
Systems  Development  (Technical  Report),  HRB-Singer,  Incor¬ 
porated,  Volume  1,  Contract  Nonr  4203(00),  Report  No. 

ND  64-68,  June  1964,  pp.  46,  (Unclassified) 

This  report  outlines  an  initial  attack  upon  the  problem 
of  determining  when,  where,  and  how  human  factors  inputs 
should  be  provided  during  the  planning  and  development 
cycle  of  Naval  Systems,  with  particular  emphasis  being 
given  to  measuring  the  consequences  of  such  inputs  in 
terms  of  systems  effectiveness.  All  work  was  directed 
toward  the  Test  and  Evaluation  (T  and  E)  phase  of  system 
development  in  order  to  develop  measures  of  effectiveness 
capable  of  being  applied  to  a  specific  system  undergoing 
T  and  E  by  the  Operational  Test  and  Evaluation  Force 
(OPTEVFCR). 

Moores,  B.  L. ,  User's  Manual  for  the  Computerized  Electronic 
System  Cost  Model;  7030  Modifications,  MITRE  Corporation, 
Bedford,  Massachusetts,  October  1964,  (Unclassified) 

A  computerized  electronic  system  cost  model  was  developed 
by  the  systems  analysis  department  and  documented  in 
ESD-TDR-63-446 .  The  present  document  describes  the 
changes  which  resulted  from  converting  the  computer 
program  for  the  model  from  FORTRAN  II  for  the  IBM  7090 
to  FORTRAN  IV  for  the  IBM  7030,  and  from  implementing 
several  operational  improvements. 


57. 

(A) 

(B) 

The  two  principal  ways  of  conducting  a  study  in  terms  of 
system  effectiveness  and  cost  are  described:  (l)  the 
"fixed  effectiveness"  case  where,  for  a  specified  level 
of  weapon  effectiveness  (e.g.,  target  destruction),  the 
alternative  equipment  possibilities  are  compared  on  the 
basis  of  economic  resource  cost,  usually  expressed  in 
terms  of  dollars  as  a  common  denominator  (2)  the  "fixed 
budget"  case  where,  for  a  specified  military  budget  for 
a  particular  mission,  the  alternative  weapon  proposals 
are  compared  on  the  basis  of  weapon  system  effective¬ 
ness,  measured,  for  example,  in  terms  of  targets 
destroyed. 

58  .  Novick,  David,  System  and  Total  Force  Cost  Analysis,  The 

(A)  Rand  Corporation,  Memorandum  RM-2695-PR,  April  1961,  pp.  l4l, 
(Unclassified) 

This  memorandum  described  the  aims,  concepts,  and  methods 
of  military  cost  analysis  as  developed  by  „he  Cost 
Analysis  Department  of  the  Rand  Corporation.  Like  the 
1956  report  on  Weapon-System  Cost  Methodology,  which  it 
replaces,  the  present  memorandum  is  concerned  basically 
with  the  estimation  of  costs  for  proposed  military 
activities  so  that  informed  choices  can  be  made  among 
them.  It  extends  the  earlier  report  by  considering  in 
more  detail  the  underlying  principles  of  cost  analysis, 
and  by  describing  methods  for  analyzing  the  costs  of 
total  force  structures  as  well  as  individual  systems. 


Novick,  David,  Concepts  of  Cost  for  Use  in  Studies  of 
Effectiveness,  Rev,  ed.,  The  Rand  Corporation,  P-.1182, 
4  October  1957 >  PP*  1^,  (Unclassified) 


59*  Odom,  Charles  T.;  &  Sebastiani,  Lambert  J.,  A  Method  of 
(A)  Considering  Attrition  Rate  in  Cost-Effectiveness  Studies, 
Ballistic  Research  Laboratories,  Aberdeen  Proving  Ground, 
Maryland,  September  1964,  (Unclassified) 


This  report  preeents  a  method  of  considering  attrition 
rate  in  cost-effectiveness  studies.  The  method  is 
developed  and  illustrated  by  its  application  to  a 
typical  situation  in  order  to  show  the  effect  of 
attrition  rate  on  missile  and  launcher  requirements. 

The  total  number  of  launchers  required  to  defeat  a  threat 
is  expressed  as  a  function  of  length  of  battle  and 
attrition  rate.  The  total  number  of  missiles  required 
to  defeat  a  threat  is  expressed  as  a  function  of 
attrition  rate.  Total  costs  then  can  be  determined  for 
each  weapon  system  reflecting  the  added  cost  of  attrition. 
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60.  Quadc,  E.  S. ,  Analysis  for  Military  Decisions,  The  Rand 

(A)  Corporation,  Santa  Monica,  California,  November  1964, 
(Unclassified) 

Contents:  Analysis  for  Air  Force  decisions,  the  selec¬ 
tion  ard  use  of  strategic  air  bases,  the  why  and  how  of 
mode?  building,  the  relevance  of  costs,  analysis  and 
design  of  conflict  systems,  assumptions  about  enemy 
behavior,  gaming  methods  and  applications,  strategies 
for  development,  mathematics  and  systems  analysis,  the 
use  of  :omputers,  costing  methods. 

61.  Report  on  Bureau  of  Naval  Personnel  Symposium,  "U.  S.  Navy 

(B)  lianpower  Management  Prospectus  for  the  1970 'sl!,  U.  S.  Naval 

(C)  Station  Washington  Navy  Yard  Annex,  16-17  April  1963, 
pp.  106,  (Unclassified) 

A  two-day  symposium,  sponsored  by  the  Bureau  of  Naval 
Personnel,  arose  out  of  a  need  for  representatives  of 
all  Navy  commands,  activities,  and  agencies  to  anticipate 
and  to  explore  Navy  manpower  requirements  in  the  1970 's. 
Manpower  requirements  were  examined  as  well  as  major 
problem  areas  to  be  encountered  in  fulfilling  these  re¬ 
quirements.  Possible  solutions  to  manpower  problems  were 
advanced,  in  the  form  of  recommendations,  by  the  seminar 
group 

62.  Rigney,  Joseph  W.;  &  Hoffman,  lyle  S.,  Human  Factors  Research 

(B)  in  Electronics  Maintenance:  An  Analysis  of  Recent  Trends, 

(C)  With  Some  Suggestions  for  the  Future,  Department  of  Psychology, 
University  of  Southern  California,  Technical  Report  No.  35, 

July  1962,  pp.  168,  (Unclassified) 

The  following  six  broad  areas  are  identified,  and  repre¬ 
sentative  studies  in  each,  along  with  their  implications, 
are  discussed: 

1.  The  measurement  of  maintainability  of  equipment  now 
in  the  field. 

2.  Military  standards  and  specifications,  and  human 
factors  handbooks  to  guide  design  of  equipment  for 
maint  ainabi lity . 

3.  The  automation  0 f  maintenance  tasks,  by  means  of 
automated  test  equipment. 

4.  Studies  of  maintenance  variables  using  the  classical 
experimental  methods. 
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5.  Investigations  in  traditional  personnel  areas  con¬ 
ducted  in  the  maintenance  field. 

6.  Mathematical  modeling  techniques  used  to  simulate 
human  problem  solving  and  other  complex  behavioral 
processes. 

63.  Schaeffer,  K.  H. ;  Fink,  John  B.;  Rappaport,  Maurice; 

(B)  Wainstein,  Leonard;  &  Erickson,  Charles  J.,  The  Knowledgeable 

(C)  Analyst:  An  Approach  to  Structuring  Man-Machine  Systems, 
Stanford  Research  Institute,  Project  No.  IMU-3546  (AIR  FORCE 
TECHNICAL  REPORT  AFOSR  4490),  February  1963,  PP-  132, 
(Unclassified) 

In  an  attempt  to  evolve  a  general  method  for  system 
analysis,  this  paper  presents  the  matrix — network  approach 
for  the  analysis  of  complex  man-machine  systems.  This 
approach  consists  of  seven  steps  which  show  how  a  system 
can  be  structured  and  how  mathemuuical  models  of  systems 
aspects  can  be  incorporated  into  the  overall  analysis. 
However,  some  of  these  steps  involve,  besides  formal 
rules,  the  judgement  of  knowledgeable  analysts.  To  delve 
deeper  into  this  judgement  function,  various  logical, 
methodological,  and  psychological  aspects  concerning 
this  function  are  discussed  '  •  different  authors.  On 
the  basis  of  these  discussion  the  principal  author 
develops  requirements  which  mut u  be  met  by  successful 
approaches  to  the  structuring  of  complex  systems. 

64.  SEA  HAWK  SHIP  CONTROL  GROUP,  SEA  HAWK  Preliminary  Cost- 

(B)  Effectiveness  Study  on  Ship  Control  Subsystems,  MEL  Research 
and  Development  Report  53/64,  22  May  1964,  (Unclassified) 

"Conducted  to  evaluate  the  broad  range  of  ship  control 
systems  available  with  the  state-of-the-art  to  determine 
the  area  for  final  system  development  to  be  employed  on 
the  SEA  HAWK  ship.  System  objectives  were  defined.  The 
selected  systems  were  divided  into  three  subsystems 
(Heading  Control,  Speed  Control,  and  Environment  Informa¬ 
tion),  and  ideal  subsystems  were  defined  for  each  objective. 
Subsystem  configuration*,  were  devised,  and  their  ability  to 
meet  the  objectives  were  rated  (Effectiveness).  Each 
subsystem  cost  was  estimated,  and  the  subsystems  were 
combined  into  six  ship  control  systems.  The  subsystem 
costs  ’.ere  sw_jed,  And  the  effectiveness  of  the  system  was 
derived.  The  cost-effectiveness  of  each  system  was  com¬ 
pared  to  an  ideal  system." 
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Sebring,  H.  C.,  Cost  Effectiveness  Logic  for  Missile  Weapon 
Systems  Selection,  General  Electric  Company,  Philadelphia, 
Pennsylvania,  July  1962,  (Unclassified) 

The  logic  for  an  effectiveness  comparison  of  missile 
weapon  systems  is  presented.  The  measure  is  defined  as 
the  number  of  a  selected  target  set  that  can  be  neutral¬ 
ized  with  a  fixed  probability  by  each  weapon  system,  of 
which  the  gross  foice  is  determined  from  a  selected 
fixed  budget.  An  important  part  of  the  logic  is  target 
destruction  assessment,  and  mathematical  methods  are 
presented  to  determine  the  numbers  of  hard  and  soft 
targets  that  can  be  neutralized  by  the  net  forces  of 
each  weapon  system.  A  basis  is  provided  for  weapon 
system  selection  since  each  can  be  evaluated  in  terms 
of  numbers  of  potentially  destroyed  targets.  A  separate 
investigation  of  the  warhead  requirements  for  area 
targets  is  presented  as  an  appendix. 

Sebring,  H.  C.,  Missile  Weapons  Systems  Effectiveness  Logic 
(U) ,  Presented  at  the  19th  National  Meeting,  Operations 
Research  Society  of  America,  25-26  May  1961,  (Unclassified) 

The  logic  for  an  effectiveness  comparison  of  missile 
weapons  systems  is  presented.  The  effectiveness' 
measure  is  defined  as  the  gross  force  cost  to  destroy 
selected  numbers  and  categories  of  targets.  Each  with 
a  fixed  probability.  Mathematical  methods  are  presented 
to  determine  the  net  and  gross  force  requirements  to 
attack  hard  point  targets  and  soft  area  targets.  With 
appropriate  modifications,  the  logic  and' methodology  are 
applicable  to  other  weapons  systems  types.  A  basiB  is 
also  provided  for  optimum  target  assignments  among 
weapons  systems  since  each  theoretical  attack  can  be 
evaluated  in  terms  of  cost. 

Sharkey,  E.  H.,  The  Advantages  of  Functional  Packaging  of 
Electronic  Equipment ,  The  Rand  Corporation,  P-1&29,  5  March 
1959 >  PP*  8,  (Unclassified) 

The  discussion  here  is  concerned  primarily  with  how 
packaging  methods  can  materially  affect  the  out-of¬ 
commission  time  of  the  aircraft,  and  the  cost  and 
difficulties  of  line  maintenance. 

The  methods  that  the  line  maintenance  people  must  use 
are  largely  dictated  by  the  decisions  made  by  the 
packaging  engineer  in  the  original  equipment  design. 
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68.  Siegel,  Arthur  I.;  &  Wolf  J.  Jay,  A  Technique  for  Evaluating 

(B)  Man-Machine  System  Designs,  Human  Factors,  Volume  3,  March  1961. 

(C)  pp.  18-28,  (Unclassified) 

"A  computer  based  method  for  digitally  simulating  the 
performance,  in  one-operator  systems,  of  operators  who 
possess  various  characteristics  is  described.  The  method 
is  believed  applicable  for  evaluating  various  system 
designs  while  the  system  is  in  the  early  design  stage. 

Two  operational  tasks,  landing  an  F^D  aircraft  on  a 
carrier  and  firing  an  air-to-air  missile,  were  simulated 
using  the  method.  The  predictions  from  the  model  were 
compared  with  outside  criterion  data  for  the  same  tasks. 

The  predictions  are  held  to  conform  generally  with 
reality  and  to  be  reasonable.  The  results  of  the  two 
applications  of  the  model  were  in  general  agreement.  It 
is  held  that  the  model  may  be  considered  sound  and  may 
now  be  tentatively  employed  for  comparative  ..valuation 
of  alternative  system  designs  or  for  predicting  system 
perf ormance . " 

69.  Siegel,  Arthur  I.;  &  Wolf,  J.  Jay,  Techniques  for  Evaluating 

(C)  Operator  loading  in  Man-Machine  Systems,  Applied  Psychological 

Services,  Contract  Nonr-2492(00),  July  1963,  pp .  59, 
(Unclassified) 

A  digital  computer  simulation  model  was  previously 
derived  and  employed  for  simulating  the  performance  of 
the  operator(s)  in  a  man- machine  system.  The  technique 
is  based  on  an  analysis  of  the  performance  of  each  opera- 
.  tor,  arranged  into  ordered,  discrete  actions  called 
"subtasks",  and  the  compilation  for  each  of  certain 
source  data. 

Since  the  development  of  the  original  model,  a  number 
of  possible  modifications  have  become  apparent.  Moreover, 
although  logical  expectancy  indicated  the  model  to  be 
usable  for  either  one  or  two  operator  systems,  the  model 
bid  never  been  exercised  on  a  uni-operator  system, 
iiccordingly,  the  present  study  focused  on  investigating: 

(l)  the  applicability  of  the  model  to  uni-operator 
situations,  and  (2)  the  effects  of  the  modifications  on 
the  agreement  of  the  results  from  application  of  the 
model  with  criterion  data. 


70. 

(C) 


71. 


72. 

(B) 

(C) 


73- 

(B) 

(C) 


74. 

(B) 


Siegel,  Arthur  I.;  Wolf,  J.  Jay,  &  Sorenson,  R.  Trent, 
Techniques  for  Evaluating  Operator  Loading  In  Man-Machine 
Systems  (Svaluation  of  One  or  a  Two-Operator  System 
Evaluative  Model  Through  a  Controlled  Laboratory  Test), 
Applied  Psychological  Services,  Wayne,  Pennsylvania, 

July  1962,  pp.  87*  (Unclassified) 

A  stochastic,  digital  computer  simulation  model  was 
previously  derived  for  and  applied  to  the  problem  of 
simulating  one  or  two  operator  man-machine  systems. 
Further  test  of  the  validity  of  the  model  through 
comparisons  of  the  model's  predictions  of  team  per¬ 
formance  with  the  actual  performance  of  trained  teams 
on  a  man-machine  task  is  described.  A  complex  two 
operator  man-machine  equipment  test  involving  team 
and  individual  branching,  operator  stress  build-up, 
looping,  cooperative  work,  communication,  waiting, 
etc.,  was  developed. 

Siegel,  B.,  Parametric  Study — Continuously  Manned  Space 
Offense  Systems  (Lj,  Aerospace  Corporation,  Los  Angeles, 
California,  January  1964,  (SECRET) 

Story,  A.  W.,  Man- Machine  System  Performance  Criteria, 
Electronic  Systems  Division,  Air  Force  Systems  Command, 

USAF,  Bedford,  Massachusetts,  ESD-TR-61-2,  May  1961,  pp.  6l, 
(Unclassified) 

In  this  report,  four  categories  of  criteria  are  developed 
for  the  evaluation  of  man-machine  system  performance. 

The  applicability  of  these  criteria  during  system  design, 
building,  and  testing  is  discussed.  Some  illustrative 
evaluations  of  man-machine  systems  are  surveyed. 

Problems  that  accrue  through  the  use  of  simulative 
features  in  the  assessment  of  system  performance  are 
considered. 

Swain,  A.  D.j&Whol,  J.  G.,  Factors  Affecting  Degree  of 
Automation  in  Test  and  Checkout  Equipment,  Dunlap  and 
Associates,  Incorporated,  Stanford,  Connecticut,  (D&A-TR- 
6O-36F),  March  1961,  (Unclassified) 

Sweet land,  Anders,  Assessing  the  Impact  of  Weapon  System 
Modifications  by  Field  Study:  A  Feasibility  Demonstration, 
The  Rand  Corporation,  Santa  Monica,  California,  Memorandum 
Report,  November  1964,  (Unclassified) 

Studies  have  been  made  on  ways  to  increase  the  produc¬ 
tivity  of  the  Air  Force  maintenance  management  system. 
These  studies  have  suggested  adding  clock  hours 


and  status  data  to  the  current  data  system  (AFM  66-1, 
Maintenance  Management),  and  have  proposed  the  use  of 
computers  in  maintenance  management  and  analysis. 

These  additions  have  made  practicable  analyses  of 
maintenance  data  that  were  heretofore  deemed  impossible. 
The  memorandum  continues  the  previous  line  of  inquiry. 

It  demonstrates  how  these  adjuncts  to  the  present  system 
may  be  used  to  assess  the  cost  and  effectiveness  of 
engineering  modifications.  The  procedures  are  described 
using  two  examples:  An  engine  modification  of  T-38A 
aircraft;,  and  a  fire-control  system  modification  of 
F-101B  aircraft.  The  two  procedures  are  then  contrasted 
to  show  how  the  combination  of  methods  can  provide  the 
most  meaningful  set  of  data.  It  is  shown  how  the  method 
of  recording  and  analyzing  may  also  be  used  to  provide  a 
history  of  the  installation- check- out  process  during  this 
vital  period. 


75.  Systems  Analysis  and  Cost  Effectiveness,  Parts  I-VI  (M), 

(B)  Adjutant  General  Department  of  the  Army,  Washington,  D.  C., 

April  1964,  (Unclassified) 

Systems  analysis  is  a  systematic,  quantitative  approach 
to  the  complex  military-economic  problems  encountered  in 
the  Defense  Program.  The  systems  analysis  cost  effective¬ 
ness  technique  is  not  a  cure-all.  Nor  is  it  an  automatic 
decision-maker.  It  is  limited  by  the  quality  and  sub¬ 
jectiveness  of  the  measurable  factors  put  into  each 
alternative.  An  increased  army  systems  analysis  effort 
can  assist  in  attaining  the  best  possible  solution  to 
Army  problems  and  improving  the  justification  of  related 
Army  programs.  Even  if  one  disagrees  in  principle  with 
some  aspects  of  analysis  or  cost  effectiveness  comparisons 
still  an  understanding  is  necessary  of  these  widely-used 
principles  and  techniques . 

76.  Systems  Effectiveness,  Office  of  Naval  Material  (Systems 

(B)  Effectiveness  Branch),  January  1965,  PP>  205,  (Unclassified) 

The  main  purpose  of  this  pamphlet  is  to  provide  personnel 
of  the  Naval  Material  Support  Establishment  (NMSE)  a 
collection  of  papers  in  a  Bingle  volume  which  reflect  the 
attitude  and  philosophy  of  the  Chief  of  Naval  Material 
towards  various  aspects  of  systems  effectiveness.  It  also 
provides  a  discussion  of  the  planning,  design,  and  cost 
considerations  in  system  development  as  well  as  some 
techniques  now  being  utilized  In  the  NMSE  in  order  to 
realize  the  development  of  effective  systems. 


77*  Tallmadge,  G.  K.;  Hoffman,  Iyle  S.;  &  Cooper,  Joel  I., 

(B)  RecommendationB  for  a  Navy-Sponsored  Research  Program  in 

(C)  Electronics  Maintenance,  Thiokol  Chemical  Corporation, 
Humetrics  Division,  Contract  No.  NOp-1352,  Technical  Report 
H-TR-63-5,  October  1963,  pp.  138,  (Unclassified) 

"A  planning  study  wsb  carried  out  to  develop  a  master 
program  of  Navy- sponsored  research  in  the  field  of 
electronics  maintenance.  Based  on  an  analysis  of 
current  and  anticipated  problems  and  trends  in  main¬ 
tenance,  a  statement  of  research  needs  is  presented 
in  the  following  four  general  program  areas:  (l)  the 
human  component,  (2)  the  hardware  component,  (3)  the 
integrated  system  and  its  support  environment,  and 
(4)  the  handling  of  maintenance  information.  Then, 
within  each  major  program  area  a  five-year  sequence 
of  research  projects  is  outlined,  roughly  costed,  and 
discussed." 

78.  Thomas,  R.  E.,  et  al.,  The  Effect  of  Various  Levels  of 

(C)  Authomation  on  Human  Operator's  Performance  in  Man-Machine 
Systems,  Battelle  Memorial  Institute,  June  19&>, 

(Unclassified) 

"If  a  system  is  under-automated,  the  operators  probably 
will  be  numerous,  overloaded,  and  'connected'  ineffi¬ 
ciently.  If  a  system  is  over-automated,  the  equipment 
will  be  complex  and  expensive  and  man  will  be  relegated 
to  a  monitoring  role.  In  either  case,  system  performance 
will  be  degraded.  Conceptually,  at  least,  an  optimum 
level  of  automation  exists.  The  project  described  here 
attempts  to  develop  an  'automation  model'  for  the  function 
allocation  problem." 


79.  Weapons  Requirements:  Integrated  Maintenance  Management  for 
(B)  Aeronautical  Weapons,  Weapon  Systems,  Related  Equipment, 
Bureau  of  Naval  Weapons,  Code  Identification  1001  WR-30, 

1  May  1963,  pp.  124,  (Unclassified) 


"This  document  establishes  the  policy,  terms  and  con¬ 
ditions  governing  the  implementation  and  execution  of 
an  integrated  maintainability  and  support  program  for 
weapons,  weapon  systems  and  related  equipments  to  be 
procured  under  the  contract  in  which  this  document  is 
cited.  It  is  the  specific  intent  of  this  document  to 
charter  the  Integrated  Maintenance  Management  Team  to 
manage  the  total  Logistic  Support  Program.  Accordingly, 
this  document  is  designed  to  develop,  early  in  a  program, 
a  maintenance  plan  which  is  tailored  to  specific 
commodities  and  contracts." 
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§0.  Westerman,  Dean  P.j  &  Mathias,  Ronald  F.,  A  Cost  Model  for 

(A)  Use  in  Cost -Effectiveness  Analyses  of  Dissimilar  Weapon 

(B)  Systems,  Ballistic  Research  Laboratories,  Aberdeen  Proving 
Ground,  Maryland,  MR16G2,  IM023201A098,  September  1964, 
(Unclassified) 

A  model  is  presented  for  the  determination  of  system 
costs  in  cost-effectiveness  studies  of  dissimilar  weapon 
systems.  The  assumptions  employed  are  listed  so  that  an 
adaptation  of  the  model  may  be  made  to  fulfill  individual 
study  requirements. 

81.  Whittenburg,  John  A.,  Methodology  for  Evaluation  of  a  Man- 

(C)  Machine  Surveillance  System,  Human  Sciences  Research, 
Incorporated,  Contract  Nonr  2525(00),  Report.  No.  6, 

December  1959,  PP-  T9,  (Unclassified) 

The  purpose  of  this  paper  was  to  focus  attention  on  the 
methodological  requirements  of  a  particular  research 
study  and  the  steps  that  were  taken  to  meet  these  re¬ 
quirements.  A  statement  of  the  general  methodological 
problem  in  this  research  study  may  be  phrased  as  follows: 
to  determine  the  performance  capabilities  and  limitations 
of  the  human  as  a  component  of  a  weapon  system  not-in- . 
being. 

The  specific  problem  of  the  study  was  to  determine  the 
capabilities  and  limitations  of  the  human,  as  an  aerial 
observer,  to  acquire  combat  intelligence  information 
under  conditions  of  ground  combat  projected  for  the 
decade  1960-1970,  so  as  to  guide  the  development  of 
criterion  measures  of  aerial  observer  proficiency  and 
pinpoint  areas  requiring  training  emphasis. 

82.  WSEIAC,  Cost-Effectiveness  Optimization  (Summary,  Conclusions 

(®)  &  Recommendations),  Final  Report  of  Task  Group  IV,  Head- 

quarters,  Air  Force  Systems  Command,  Andrews  Air  Force  Base, 
Maryland,  AFSC-TR-65-4,  Volume  1,  January  1965, 

(Unclassified) 


"The  underlying  principles  associated  with  cost- 
effectiveness  analysis  are  discussed.  The  rationale, 
purpose,  methodology  required,  and  nature  of  the 
results  that  can  be  obtained  by  means  of  the  analysis 
are  presented  in  summary  form.  Illustrations  of  the 
type  of  input  data  required  and  the  logic  associated 
with  its  application  are  provided.  The  summary  con¬ 
stitutes  an  overview  of  the  more  detailed  task  analysis 
and  supplementary  technical  material  presented  in 
Volumes  II  and  III.  Included  are  the  conclusions  and 
recommendations  aB  set  forth  in  Volume  II." 


83 .  WSEIAC,  Cost-Effectiveness  Optimization  (Technical  Supplement), 
(B)  Final  Report  of  Task  Group  IV,  Headquarters,  Air  Force  Systems 
Command,  Andrews  Air  Force  Base,  Maryland,  AFSC-TR-65-4, 

Volume  III,  January  1965  (Unclassified) 

"A  discussion  of  optimization  which  amplifies  the  material 
in  Volume  II,  Section  IV  :j.s  presented.  Optimization 
principles,  criteria  and  checklists,  as  well  as  a  summary 
of  various  applicable  techniques  is  included.  A  series 
of  six  examples  are  described  covering  a  number  of  critical 
aspects  of  cost-effectiveness  analysis  in  considerable 
detail.  Treated  in  the  examples  are:  (l)  Optimization  of 
effectiveness  based  on  reliability,  maintainability, 
performance,  and  cost;  (2)  Allocation  of  reliability  re¬ 
quirements  among  subsystems;  (3)  Payload  allocation  among 
three  subsystems  based  on  a  fixed  weight  constraint; 

(4)  Determination  of  best  checkout  routine  for  a  limited 
prelaunch  test;  (5)  Optimization  of  availability  for  a 
complex  system;  and  (6)  Trade-off  study  between  site 
hardening  and  dispersal  for  a  missile  system." 

84  .  WSEIAC,  Management  Systems  (Elements  of  Effectiveness 
(B)  Assurance  Management ) ,  Final  Report  of  Task  Group  V,  Head- 
quarters,  Ai~  Force  Systems  Command,  Andrews  Air  Force  Base, 
Maryland,  AFSCR-TR-65,  Volume  II,  January  1965,  (Unclassified) 

"...  presents  summaries  of  many  of  the  studies  carried 
on  by  the  task  group  during  the  six  months  of  WSEIAC 
formal  meetings  and  investigations.  From  these  studies 
the  concept  and  philosophy  of  a  System  Effectiveness 
Assurance  Management  System  (SEAMS),  the  major  policy 
issues,  and  recommendations  presented  in  Volume  I  were 
developed.  Air  Force  management  of  system  effectiveness 
activities  is  assessed  through  surveys  of  some  of  the 
principal  offices  and  conmands.  Industry  capability 
for  response  to  these  new  requirements  is  measured.  A 
review  of  activities  and  discipline  requirements 
essential  to  effectiveness  assurance  management  in  the 
context  of  the  AF  375  series  documents  is  provided. 

Finally,  a  series  of  studies  and  discussions  of  pertinent 
elements  of  system  effectiveness  is  furnished,  including 
a  data  central  proposal,  the  effect  of  incentives,  and 
appendixes  on  specification  management,  parts  research 
and  program  management  elements  of  the  concept  arid 
definition  phases  of  system  development." 
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85 .  WSEIAC,  Requirements  Methodology,  Final  Report  of  Task  Group 
(B)  I,  Headquarters,  Air  Force  Systems  Command,  Andrews  Air  Force 
Base,  Maryland,  AFSC-TR-65-I,  January  1965,  (Unclassified) 


"The  objective  of  Task  Group  I  was  'To  review  present 
procedures  being  used  to  establish  system  effectiveness 
requirements  and  recommend  a  method  for  arriving  at 
requirements  that  are  mission  responsive'.  Applicable 
documents  were  examined,  including  Department  of 
Defense  Directives  and  Instructions,  Air  Force  Regula¬ 
tions,  Manuals,  Specifications,  Office  Instructions,  etc., 
that  might  be  used  to  establish  effectiveness  require¬ 
ments.  Detailed  examination  of  the  Specific  Operation 
Requirement  (SOR)  and  the  companion  Directorate  Office 
Instruction  (DOl)  11-7  resulted  in  the  preparation  of  a 
proposed  Air  Force  Manual  (Appendix  I) .  This  document 
provides  checklists,  guidelines,  and  procedures  for  SOR 
preparation  that  include  the  significant  elements  of 
system  effectiveness.  A  proposed  Air  Force  Regulation 
(Appendix  II )  was  developed  to  formalize  a  program  of 
effectiveness  evaluation  ana  prediction  for  the  system 
life  cycle.  Policy,  concepts,  and  major  command 
responsibilities  are  developed.  Additional  conclusions 
and  recommendations  are  submitted  relative  to  the  effec¬ 
tiveness  requirements  that  constitute  necessary  steps 
to  development  of  an  Air  Force  wide  system  effectiveness 
management  program." 
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1.  "ADM  Lee  Tells  How  Weapons  Systems  Are  Analyzed",  Data, 

(B)  Volume  10,  No.  5>  May  19^5>  PP-  35-40,  (Unclassified) 

(C) 

The  methodologies  used  and  the  problems  encountered  in 
weapon  system  analyses  are  discussed  in  this  interview. 
ADM  Lee  also  discusses  the  "man  function"  in  the  man- 
machine  relationship. 

2.  "Aid  in  Determining  Profit  on  Fee  Objective  for  a  Contract", 
(A)  The  Navy  Management  Review,  NAVEXOS  P910,  Volume  IX,  No.  7 

and  8,  July- August  1964,  PP>  15-19 >  (Unclassified) 

"Weight  Guidelines"  is  a  method  of  assigning  weights  or 
percentage  points  to  the  many  factors  which  must  be 
taken  into  account  in  determining  the  profit  or  fee 
objective  for  a  contract.  It  is  a  means  for  contracting 
officers  to  better  evaluate  cost  proposals  and  decide 
upon  a  profit  or  fee  objective  which  properly  reflects 
the  contractor's  justifiable  return.  "Weighted  Guide¬ 
lines"  is  meant  to  be  used  only  in  the  negotiation  of 
profits  where  cost  analysis  is  performed. 

3.  Amber,  G.  H.;  &  Amber,  P.  S.,  A  Yardstick  for  Automation, 

(C)  Instruments  and  Automation,  Volume  30,  1957,  PP*  677, 

(Unclassified) 

Various  techniques  have  been  used  to  classify  automation 
devices  into  basic  sub-units.  James  Bright  used  th<- 
graphic  "Automation  Profile",  Leaver  and  Brown  wrote  a 
"Functional  Morp’  ology  of  Mechanisms"  and  Amber  and  Amber 
here  propose  a  mathematical  notation  based  on  the  infor¬ 
mation  and  energy. 

4.  Arzigian  S.j  &  Clary,  J.,  "Report  on  Enlisted  Personnel 
(A)  Replacement  Costs  (BM,  FT,  MT,  RM)",  PRAW  Report  (64-4), 

February  ly64,  (Unclassified) 

This  research  report  was  designed  to  meet  the  urgent 
need  of  the  Chief  of  Naval  Personnel  for  enlisted 
personnel  replacement  costs  for  the  BM,  ET,  FT,  MT,  and 
RH  ratings.  The  reDlaeement  costs  presented  were  based 
on  tbe  latest  available  information  and  reflected  all 
known  adjustments,  including  the  increases  in  pay  and 
allowances  authorized  by  Congress  in  1963* 

5.  Arzigian  S.j  Couloumbis,  T.,  "Report  on  Enlisted  Personnel 
(A)  Replacement  Costs",  PRAW  Report  (63-8),  May  i960, 

(Unclassified) 


This  research  effort  was  conducted  in  response  to  the 
urgent  need  for  a  comprehensive  study  of  "Enlisted 
Personnel  Replacement  Cost"  for  use  by  the  Chief  of  Naval 
Personnel  in  satisfying  requests  from  higher  authorities. 
The  report  is  limited  to  a  discussion  of  the  costs 
associated  with  replacement  of  ETs,  FTs,  and  ATs  in  pay 
grades  E-l  through  £-6. 

6.  Arzigian,  S.j  Couloumbis,  T.,  "Report  on  Enlisted  Personnel 
(A)  Replacement  Costs",  PRAW  Report  (63-22),  August  1963, 

(Unclassified) 

This  report  presents  research  findings  of  the  second 
portion  of  Phase  I  of  a  study  dealing  with  enlisted 
personnel  replacement  costs .  "Report  on  Enlisted 
Personnel  Replacement  Costs,"  PRAW  Report  Number  63-8 
of  May  1963  discussed  the  replacement  costs  of  ETs,  FTs, 
and  ATs.  This  report  is  a  supplement  to  it  and  provides 
replacement  costs  for  the  following  seventeen  additional 
ratings:  HD,  SO,  MT,  RM,  TM,  MR,  CE,  MM,  MV,  EM,  HM,  BM, 
VT,  YN,  SK,  JO,  CS. 

7.  Chapanis,  A.,  Human  Engineering,  in  Operations  Research  and 
(C)  Systems  Engineering  (Ed.  by  C.  D.  Flagle;  W.  H.  Huggins  & 

R.  H.  Roy) 7  The  John  Hopkins  Press,  Baltimore,  Maryland, 
i960.  Chapter  19,  pp.  53*+- 582,  (Unclassified) 


This  chapter  covered  some  of  the  human  factors  involved 
in  the  design  of  automatic  and  semiautomatic  machine 
systems.  It  demonstrated  that  there  are  still  quite  a 
few  human  problems  in  most  such  systems,  and  that  a 
successful  automatic  system  requires  the  engineer  to 
consider  carefully  the  role  of  the  human  operator  and 
how  he  is  designed  into  the  system.  Finally,  it 
suggested  why  the  human  factors  specialist  is  often  con¬ 
sidered  a  member  of  the  systems  design  team,  and  in  what 
ways  he  can  contribute  to  the  important  and  challenging 
task  of  designing  new  systems  for  our  automatic  world 
of  tomorrow. 

8.  Chapanis,  A.,  On  Some  Relations  Between  Human  Engineering, 

(C)  Operations  Research  and  Systems  Engineering,  A  report  of 
research  r  contract  with  Office  of  Naval  Research,  John 
Hopkins  ceity,  Report  No.  8,  13  May  i960,  (Unclassified) 

This  is  an  elaboration  of  some  remarks  made  at  the  First 
Systems  Symposium  at  the  Case  Institute  of  Technology, 
Cleveland,  Ohio,  April  26-28,  i960.  Chapanis  discusses 
some  of  the  activities  and  the  relations  between  human 
engineering,  operations  research  and  systems  engineering. 


9.  Chapanis,  A.,  On  the  Allocation  of  Functions  Between  Man  and 
(C)  Machines,  Occupational  Psychology,  Volume  39*  No.  1,  1965, 
pp.  1-11,  (Unclassified) 

Describes  the  nature  of  the  allocation  problem,  discusses 
approaches  taken  to  the  problem  in  the  past,  and  gives 
some  of  the  authors  views  on  the  contemporary  status  of 
the  problem,  along  with  suggestions  of  a  strategy  for 
dealing  with  it . 

10.  Enlisted  Personnel  Procurement  and  Processing  Cost  Report, 

(A)  (DD  Form  804,  Report  Control  Symbol  DD-MP  &  R  (SA)  390), 
S2CNAV  Instruction  1130.2A,  Fiscal  Year  1963  (second  half), 
(Unclassified) 

11.  Fitts,  P.  M.,  Functions  of  Man  in  Complex  Systems,  Aerospace 
(C)  Engineering,  Volume  21,  No.  1,  pp.  34-39,  (Unclassified) 

(B) 

Fitts  states  that  "the  central  issue  in  allocating 
functions  to  men  in  systems  is  the  overall  performance 
of  the  system,  relative  to  cost,  not  what  mem  does 
'better '  than  a  machine,  or  vice  versa" .  After  dis¬ 
cussion  of  the  appropriate  way  to  approach  the  question 
of  man's  role  in  complex  systems,  some  of  his  major 
limitations  and  special  capacities  are  considered. 

11.  Grodsky,  M.  A. ,  Risk  and  Reliability,  Aerospace  Engineering, 

(C)  Volume  21,  No.  1,  1962,  pp.  28-33,  (Unclassified) 

(B) 

This  paper  was  primarily  concerned  with  presenting  data 
demonstrating  the  contributions  of  a  spacecraft  crew  to 
higher  reliability  and  lower  risk. 

The  performance  of  the  crew  in  terms  of  efficiency  and 
reliability  was  discussed  under  two  experimental  con¬ 
ditions.  The  ability  of  the  human  within  these  situa¬ 
tions  appeared  adequate.  A  brief  discussion  of  stress 
was  also  given  which  indicated  that  sensory  deprivation 
effects  can  be  reduced  by  selection,  training,  and 
spacecraft  arrangement. 

12.  "How  We  Buy  and  What  We  Buy",  The  Navy  Management  Review, 

(A)  NAVEXOS  P910,  Volume  IX,  No.  7  and  8,  .July- August 1964, 

pp.  4-6,  (Unclassified)  . 


This  article  takes  a  look  at  the  procurement  functions 
and  the  various  procurement  programs  undertaken  by  the 
Office  of  Naval  Material. 


13.  Jackson,  R.;  Mann,  L.;  &  Primes,  W.,  Officer  Personnel  Costs 

(A)  For  Use  In  ASW  Surface  Ship  Systems  Cost  Effectiveness 

(B)  Comparisons,  PRAW  Report  (65-63).  June  i965,  (Unclassified) 

This  memorandum  presents  costs  for  officers  estimated  to 
man  an  ASW  surface  ship  (DE  type).  However,  costs  are 
considered  appropriate  for  other  surface  type,  (1100), 
officers.  Officer  cost  data  presented  are  designed  for 
use  in  cost  effectiveness  studies  for  projects  in  the 
RDT&E  cycle. 

14.  Morton,  A.  S.,  et  al.,  Annex  B  -  A  Survey  Study  of  Reenlist- 

(A)  ment  Incentives,  INS  Study  II  -  Manning  the  Future  Navy  (U), 

NOnr  -  3732(00),  October  1964,  (Unclassified) 

The  primary  aim  of  this  study  of  reenlistment  and  in¬ 
centives  was  to  obtain  information  about  the  helpfulness 
of  possible  new  incentives  in  raising  the  first  term 
enlisted  reenlistment  rates  for  personnel  categories 
where  these  rates  are  undesirably  low. 

15.  Navy  Department,  Cost  Estimates  of  Weapons  Systems  Ships, 
Aircraft  and  Task~Forces  -  Fiscal  Year  1962  (U),  NAVEXOS 
PI986,  Office  of  the  Comptroller,  31  March  1961, 

(CONFIDENTIAL) 

This  publication  sets  forth  statistical  cost  estimates 
developed  for  overall  planning  purposes  by  the  Performance 
Analysis  Branch  of  the  Progress  Reports  and  Statistics 
Division.  These  direct  costs  for  ships,  aircraft  and 
guided  missiles  are  presented  to  serve  as  a  general 
reference  for  Navy  use.  It  is  the  fourth  edition  of 
"COST"  prepared  by  the  Comptroller  of  the  Navy  and  data 
are  based  on  1962  budget  factors. 

16.  Per  Captia  Cost  of  Training  in  Navy  Schools  (BUPERS  Management) 

(A)  BUPERS  Notice  1500,  Fiscal  Year  1963,  (Unclassified) 

17.  Stroud,  J.;  &  Irwin,  W.,  "Protect  the  Military  Victims  of 

(C)  Automated  Command  Systems",  Armed  Forces  Management,  Volume  11, 

(B)  No.  10,  July,  1965,  pp.  87-89,  (Unclassified) 

Using  the  Naval  Tactical  Data  System  (NTDS)  as  an  example, 
the  authors  contend  that,  as  the  automated  command  "tree" 
becomes  more  efficient,  it  will  begin  to  displace  people 
in  the  traditional  command  structure.  And  as  this  happens, 
the  authors  say,  our  attitude  toward  personnel  retraining 
will  have  to  change.  Rather  than  a  sought  for  privilege, 
retraining  will  have  to  become  a  well  paid  duty. 


18.  "Systems  Effectiveness:  OEMs  Approach" ■,  Data,  Volume  10, 
(B)  No.  5»  May  19^5>  PP*  35-^0 >  (Unclassified) 

This  article  contains  an  interview  with  Commander  Keith 
N.  Sargent,  DSN,  concerning  "Systems  Effectiveness"  and 
related  topics. 
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